Deep learning image processing method for determining vehicle damage

ABSTRACT

In a computer-implemented method and associated tangible non-transitory computer-readable medium, an image of a damaged vehicle may be analyzed to generate a repair estimate. A dataset populated with digital images of damaged vehicles and associated claim data may be used to train a deep learning neural network to learn damaged vehicle image characteristics that are predictive of claim data characteristics, and a predictive similarity model may be generated. Using the predictive similarity model, one or more similarity scores may be generated for a digital image of a newly damaged vehicle, indicating its similarity to one or more digital images of damaged vehicles with known damage level, repair time, and/or repair cost. A repair estimate may be generated for the newly damaged vehicle based on the claim data associated with images that are most similar to the image of the newly damaged vehicle.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/023,414, filed Jun. 29, 2018, and entitled “DEEP LEARNING IMAGEPROCESSING METHOD FOR DETERMINING VEHICLE DAMAGE;” which claims priorityto and the benefit of: U.S. Application No. 62/526,879, filed Jun. 29,2017, and entitled “DEEP LEARNING IMAGE PROCESSING METHOD FORDETERMINING VEHICLE DAMAGE;” the entire disclosures of each of which arehereby incorporated herein in their entirety.

FIELD OF THE DISCLOSURE

The present disclosure generally relates to technology for estimatingdamage level, repair time, and/or repair cost for a damaged vehicle.

BACKGROUND

An insured client whose vehicle has recently been damaged may wish toquickly obtain an initial estimate of the damage level, repair time,and/or repair cost for his or her damaged vehicle. Often, obtaining suchan estimate may require that the insured client take the damaged vehicleto a repair shop. This may be difficult and time consuming for variousreasons. For example, the extent of the damage to a vehicle may make itdifficult for the client to transport the vehicle to the repair shop, orthe nearest repair shop may be located far away. Even if a client isable to take his or her damaged vehicle to a repair shop, he or she maywish to see a second opinion to confirm the damage level, repair time,and/or repair cost for his or her damaged vehicle.

From past submitted insurance claims, insurance companies may have largedatabases containing images of damaged vehicles from a variety ofperspectives, and associated claim data. In each instance, the claimdata may include the damage level, repair time, and/or repair cost forthe vehicles shown in the images. However, this rich source ofinformation is generally not utilized once the insurance claims at issueare settled.

SUMMARY

The present embodiments disclose systems and methods that may relate to,inter alia, estimating damage level, repair time, and/or repair cost fora damaged vehicle by using deep learning image processing to compare adigital image of the damaged vehicle to one or more digital images ofother damaged vehicles for which the damage level, repair time, and/orrepair cost is known.

In one aspect, a computer-implemented method is provided that mayinclude accessing a historical training dataset populated with digitalimages of damaged vehicles and historical claim data related to damagedvehicles and generating a predictive similarity model by using thehistorical training dataset to train a deep learning neural network.Training the deep learning neural network may include learning thedamaged vehicle image characteristics that are predictive of one or moreof damage level, repair time, or repair cost, as well as the degree towhich each of the damaged vehicle image characteristics is predictive ofone or more of damage level, repair time, or repair cost. The method asprovided may further include determining, using the predictivesimilarity model, a similarity score for a given two digital images ofdamaged vehicles, and determining, based on the similarity score, thatthe given two digital images satisfy one or more similarity criteria. Inresponse to determining that the two digital images satisfy the one ormore similarity criteria, the method may further include displaying thetwo digital images on a single page and predicting the damage level,repair time, or repair cost for a first vehicle of the damaged vehiclesbased on the damage level, repair time, or repair cost for a secondvehicle of the damaged vehicles.

In another aspect, a tangible, non-transitory computer-readable mediumis provided that may store instructions that, when executed by one ormore processors, cause the one or more processors to access a historicaltraining dataset populated with digital images of damaged vehicles andhistorical claim data related to damaged vehicles and generate apredictive similarity model by using the historical training dataset totrain a deep learning neural network. The instructions to train the deeplearning neural network may include instructions that cause the one ormore processors to learn the damaged vehicle image characteristics thatare predictive of one or more of damage level, repair time, or repaircost, as well as the degree to which each of the damaged vehicle imagecharacteristics is predictive of one or more of damage level, repairtime, or repair cost. The tangible, non-transitory computer-readablemedium as provided may further include instructions that cause the oneor more processors to determine, using the predictive similarity model,a similarity score for a given two digital images of damaged vehicles,and determine, based on the similarity score, that the given two digitalimages satisfy one or more similarity criteria. In response to adetermination that the two digital images satisfy the one or moresimilarity criteria, the instructions may further cause the one or moreprocessors to display the two digital images on a single page andpredict the damage level, repair time, or repair cost for a firstvehicle of the damaged vehicles based on the damage level, repair time,or repair cost for a second vehicle of the damaged vehicles.

In another aspect, a computer system includes one or more processors anda memory storing instructions that, when executed by the one or moreprocessors, cause the one or more processors to access a historicaltraining dataset populated with digital images of damaged vehicles andhistorical claim data related to damaged vehicles. The instructions alsocause the one or more processors to generate a predictive similaritymodel by using the historical training dataset to train a deep learningneural network. Training the deep learning neural network includeslearning the damaged vehicle image characteristics that are predictiveof one or more of damage level, repair time, or repair cost, and thedegree to which each of the damaged vehicle image characteristics ispredictive of one or more of damage level, repair time, or repair cost.The instructions also cause the one or more processors to determine,using the predictive similarity model, a similarity score for a giventwo digital images of damaged vehicles, determine, based on thesimilarity score, that the given two digital images satisfy one or moresimilarity criteria, and, in response to a determination that the twodigital images satisfy the one or more similarity criteria, display thetwo digital images on a single page. The instructions also cause the oneor more processors to predict the damage level, repair time, or repaircost for a first vehicle of the damaged vehicles based on the damagelevel, repair time, or repair cost for a second vehicle of the damagedvehicles.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts a system for estimating damage level, repair time, and/orrepair cost for a damaged vehicle.

FIG. 2 is a data flow diagram depicting an exemplary image search enginefor selecting and displaying images similar to an image of a damagedvehicle.

FIG. 3 is a flow diagram of an exemplary method for estimating thedamage level, repair time, and/or repair cost for a damaged vehicle.

FIG. 4 depicts an exemplary set of user interfaces for uploading animage of a damaged vehicle, comparing the image to similar images ofother damaged vehicles, and displaying a repair estimate for the damagedvehicle.

DETAILED DESCRIPTION

The embodiments described herein relate to, inter alia, systems andtechniques for analyzing an image of a damaged vehicle to estimate adamage level, repair time, and/or repair cost for the damaged vehicle.An insured client, for instance, may take a digital photo of a damagedvehicle using a mobile device (e.g., a smartphone) and, using thesystems and techniques described herein, obtain a repair estimate forthe damage depicted in the digital image. The repair estimate may bebased on the similarity of a damaged vehicle image to other damagedvehicle images for which a damage level, repair time, and/or repair costare known. For example, the known damage level, repair time, and/orrepair cost for similar images may be averaged or otherwisestatistically analyzed to provide a repair estimate.

To obtain the damage and/or repair estimate, an image search engine maylocate similar images by analyzing a database of images, and generatingsimilarity scores for the damaged vehicle image and each of a number ofother images in the database. The similarity scores may be generatedaccording to a predictive similarity model. In particular, thepredictive similarity model may be a set of rules indicating imagecharacteristics predictive of damage level, repair time, and/or repaircost, and may be derived from a deep learning analysis of historicaldamaged vehicle images and associated insurance claim data. Two damagedvehicle images sharing several predictive image characteristicsindicated by the predictive similarity model, for example, may yield ahigher similarity score than two damaged vehicle images with fewerpredictive image characteristics in common. The image search engine mayselect a number of images with the highest similarity scores to be usedfor the repair estimate.

In some embodiments, a user interface may be configured to allow a userto upload a photo of his or her damaged vehicle and view images ofsimilar damaged vehicles as determined and selected or filtered by theimage search engine. The user may have the option of selecting some ofthe vehicles to be averaged or otherwise statistically analyzed toprovide a repair estimate, for example, and the repair estimate may thenbe displayed to the user.

FIG. 1 depicts an exemplary system 100 for determining an estimateddamage level, repair time, and/or repair cost for a damaged vehicle. Thesystem 100 may include a damaged vehicle 102, a client computing device104, a server 106, and a wired and/or wireless network 108. Although thedamaged vehicle 102 is depicted as a car in FIG. 1 , it will beunderstood that the damaged vehicle 102 may be any type of vehicle, suchas, for example, a bus, truck, motorcycle, all-terrain vehicle,snowmobile, jet ski, golf cart, boat, or aircraft. The damaged vehicle102 may be a recently damaged vehicle, for which a preliminary estimateof repair cost, repair time, and/or another, less quantitativeindication of damage level (e.g., “heavy damage,” etc.) is desired. Thedamaged vehicle 102 may be owned by an insured client (e.g., anindividual customer, a rental car company, etc.).

The client computing device 104 may be any suitable device with a memory110 and one or more processors (not shown in FIG. 1 ), such as asmartphone, a laptop computer, a desktop computer, a tablet, or awearable device, for example. The memory 110 may be a computer-readable,non-transitory storage unit or device, or collection of units/devices,and may include persistent (e.g., hard disk) and/or non-persistentmemory components. The memory 110 may store instructions that areexecutable on one or more processors of the client computing device 104to perform various operations, including the instructions of varioussoftware applications and the data that is generated and/or used by suchapplications.

The client computing device 104 may also include a camera 112, which maybe used to take a photograph producing a digital image 114 of thedamaged vehicle 102. In some embodiments, the camera 112 may be separatefrom the client computing device 104, and may send the damaged vehicleimage 114 to the client computing device 104 by a wired or wirelessnetwork (e.g., a Bluetooth network, a wireless local area network(WLAN), a cellular network, etc.). In one embodiment, for example, thecamera 112 is integrated in a smartphone, while the client computingdevice 104 is a laptop or desktop computer that communicates with thesmartphone via Bluetooth or a WLAN. Additionally, the client computingdevice 104 may send the damaged vehicle image 114 to the server 106 viathe network 108.

The server 106 may include one or more processors (not shown in FIG. 1 )and a memory (also not shown in FIG. 1 ). The memory may be acomputer-readable, non-transitory storage unit or device, or collectionof units/devices, and may include persistent (e.g., hard disk) and/ornon-persistent memory components. The memory may store instructions thatare executable on the one or more processors to perform variousoperations, including the instructions of various software applicationsand the data that is generated and/or used by such applications. Thestored instructions may include instructions for executing a predictivesimilarity model generator 116 and an image search engine 118.

The predictive similarity model generator 116 may incorporate a deeplearning neural network, or any other suitable kind of machine learning,to learn patterns in image characteristics from damaged vehicle imagesfor which claim data is known. In particular, the predictive similaritymodel generator 116 may access a historical training dataset 120 tolearn image characteristics that are predictive of various claim datacharacteristics (e.g., damage level, repair time, repair cost, or anyother claim data characteristics relating to damage and/or repair).Based on the learned image characteristics, the predictive similaritymodel generator 116 may generate a predictive similarity model, whichmay include data and/or rules indicating which image characteristics arerelevant for predicting damage and/or repair of a vehicle depicted inthe image.

The historical training dataset 120 may include a set of historicalimages of damaged vehicles 122, 124, 126 and a set of historical claimdata 128, 130, 132, from which the predictive similarity model generator116 may be trained. Each of the historical images 122, 124, 126 maydepict the damaged vehicles from a particular perspective, and may beassociated with particular insurance claim data. In the exampleembodiment and scenario shown in FIG. 1 , historical image 122 isassociated with historical claim data 128, historical image 124 isassociated with historical claim data 130, and historical image 126 isassociated with claim data 132. Each of historical claim data 128, 130,132 may include data (e.g., text, field values or codes, etc.)indicating information such as how a vehicle was damaged, where avehicle was damaged, the qualitative severity of damage to a vehicle(e.g., “heavy,” “moderate,” etc.), the repair time for the damage to avehicle, the repair cost for the damage to a vehicle, and/or any otherinformation relevant to an insurance claim.

Although the depicted historical training dataset 120 includes threehistorical images 122, 124, 126 and three historical claim data 128,130, 132, other embodiments and/or scenarios may include more or fewerimages and more or fewer claim data (e.g., thousands of images andthousands of respective sets of claim data). Moreover, while FIG. 1shows one historical image 122, 124, 126 associated with each ofhistorical claim data 128, 130, 132, some embodiments may include aplurality of historical images associated with each of historical claimdata 128, 130, 132. For example, historical image 122 may includemultiple images of the same damaged vehicle, all associated withhistorical claim data 128. The multiple images may include a first imageshowing a car that suffered a front impact from a driver-sideperspective (i.e., a profile view), and a second image showing the samecar from a frontal perspective, for instance.

By processing the historical image 122 associated with historical claimdata 128, the historical image 124 associated with claim data 130, thehistorical image 126 associated with historical claim data 132, and soforth, the predictive similarity model generator 116 may learn whichimage characteristics are indicative or predictive of which claim datacharacteristics (e.g., indicative or predictive of damage level, repairtime, repair cost, and/or any other claim data characteristics relatingto damage and/or repairs). For images that show a bent bumper portion ofa vehicle, for example, the predictive similarity model generator 116may determine that the depicted degree of bending within a particulararea of the bumper is predictive of damage levels, repair times, and/orrepair costs. The predictive similarity model generator 116 may also useinformation from the sets of historical claim data 128, 130, 132 toincorporate a greater level of specificity into the model. For example,the generated model may separately specify, for each of a number ofdifferent vehicle makes and models, the correlation between differentranges of bending in the bumper and different average repair times,costs, etc.

Furthermore, the predictive similarity model generator 116 may learn thedegree to which each of the damaged vehicle image characteristics isindicative or predictive of certain claim data characteristics. Forexample, the predictive similarity model generator 116 may determinethat a depicted degree of bending within a particular area of the bumperis highly predictive of repair time and/or repair cost. Bumpers bentbeyond a certain angle may almost invariably require a replacement, forinstance, or may even be highly predictive of frame or enginecompartment damage. As another example, images that show scratching ormarks on exterior portions of a bumper (or a vehicle generally) may beonly slightly predictive of repair time and/or repair cost.Marks/scratches may be less predictive because it tends to be unclear,from an image alone, how deep a “scratch” may be and, therefore, whetherthe scratches or marks can easily be buffed out or instead requirecomplete replacement (and/or some other more complex method of repair).Therefore, the costs associated with these image characteristics may behighly variable, and the image characteristics may only be predictive ofa very wide range of potential costs and/or repair times.

Using the predictive similarity model, the image search engine 118 maylocate one or more images with a high level of predictive similarity tothe damaged vehicle image 114, to be used for a repair estimate or otherdamage level indicator. The image search engine 118 may include asimilarity score generator 134 and a comparison image selector 136. Thesimilarity score generator 134 may access the damaged vehicle image 114and a damaged vehicle digital image dataset 138 that includes images forwhich repair data (and/or other claim data characteristics relating todamage level) is known. The similarity score generator 134 may then usethe rules generated by the predictive similarity model generator 116 togenerate a similarity score for each image in the damaged vehicledigital image dataset 138, with the score indicating the similaritybetween the image and the damaged vehicle image 114. Using thesimilarity scores, the comparison image selector 136 may select imagesfrom the dataset 138 to be compared with the damaged vehicle image 114for the repair estimate or other damage level indicator.

The digital image dataset 138 may include a set of images 140, 142, 144depicting damaged vehicles and a set of associated claim data 146, 148,150. Each of the images 140, 142, 144 may depict a damaged vehicle froma particular perspective, and may be associated with particularinsurance claim data. For example, as shown in FIG. 1 , image 140 isassociated with claim data 146, image 142 is associated with claim data148, and image 144 is associated with claim data 150. Moreover, theclaim data 146, 148, 150 may include information similar to thatdescribed above in connection with historical claim data 128, 130, 132.

Although as depicted in FIG. 1 the damaged vehicle digital image dataset138 includes three images 140, 142, 144 and three claim data 146, 148,150, other embodiments and/or scenarios may include any number of imagesand/or claim data. Additionally, although FIG. 1 shows one image 140,142, 144 associated with each of claim data 146, 148, 150, someembodiments may include a plurality of images (e.g., showing differentperspectives of a same damaged vehicle) associated with each of claimdata 146, 148, 150. Moreover, while the damaged vehicle digital imagedataset 138 is depicted in FIG. 1 as distinct from the historicaltraining dataset 120, they may be the same dataset, or may include atleast some of the same images and claim data, e.g., at least one ofimages 140, 142, 144 may be at least one of historical images 122, 124,126, and at least one of claim data 146, 148, 150 may be at least one ofhistorical claim data 128, 130, 132.

When comparing the images 140, 142, 144 with the damaged vehicle image114, the similarity score generator 134 may use the rules, algorithm,and/or data of the predictive similarity model to detect which damagedvehicle image characteristics, previously identified as being predictiveof claim data characteristics, are present in the damaged vehicle image114. The similarity score generator 134 may then determine which imageor images in damaged vehicle digital image dataset 138 share one, some,or all of the same characteristic(s). For each image in dataset 138 thatshares at least one such characteristic, similarity score generator 134may generate a similarity score. The score calculation may involvecounting the number of common image characteristics, and/or applying anyweighting of particular image characteristics as specified by thepredictive similarity model, for example.

For example, if damaged vehicle image 114 has an image characteristicthat is strongly predictive of a particular claim data characteristic,the similarity score generator 134 may tend to generate a highersimilarity score for any images within dataset 138 that share that imagecharacteristic. On the other hand, if damaged vehicle image 114 has animage characteristic that is only weakly predictive of a particularclaim data characteristic, the similarity score generator 134 may tendto generate a lower similarity score for any images within dataset 138that share that image characteristic. Of course, similarity scores maybe lower still (e.g., zero) for images within dataset 138 that do notexhibit the image characteristic at all.

The similarity score generator 134 may generate a similarity score forthe damaged vehicle image 114 and each of the images 140, 142, 144 inthe damaged vehicle digital image dataset 138. That is, each similarityscore may be associated with a particular image 140, 142, 144. It shouldbe appreciated the similarity score generator 134 may generate more orfewer similarity scores based on the number of images 140, 142, 144 inthe damaged vehicle digital image dataset 138, with each similarityscore corresponding to an image in the damaged vehicle digital imagedataset 138. Alternatively, the similarity score generator 134 maygenerate a similarity score only with respect to those images withindataset 138 that share at least one (predictive) image characteristic incommon with damaged vehicle image 114.

The comparison image selector 136 may access the similarity scores todetermine whether each of images 140, 142, 144 (or each of a subset ofthose images) falls above or below a similarity score threshold, orotherwise satisfies some suitable similarity criteria for use in arepair and/or damage level estimate. If a threshold is used, thecomparison image selector 136 may use a similarity score threshold thatis preset, or a similarity score threshold that is selected by a user.In some embodiments, the comparison image selector 136 may rank thesimilarity scores from highest to lowest in order to select a certainnumber of most similar images, where the number of images may be fixed(e.g., 20 images) or selected by a user.

The server 106 may send the images selected by the comparison imageselector 136 to the client computing device 104 via the network 108. Theclient computing device 104 may then display the images on a userinterface, such as the user interface 404 discussed below with respectto FIG. 4 . In some embodiments, the client computing device 104, viathe user interface, may prompt a user to select one or more of thedisplayed images that show damage that looks visually most similar tothe damage to damaged vehicle 102. Based on the selection of the user,the client computing device 104 may display claim data corresponding tothe selected image(s) (or other information derived therefrom, such asan average repair cost/time, etc.) via a user interface such as userinterface 406 discussed below with respect to FIG. 4 .

For instance, if the user selects image 140 as most visually similar,then some or all of claim data 146 corresponding to image 140 may bedisplayed via the user interface. If the user selects a plurality ofimages, the client computing device 104 may generate an average or otherstatistical analysis of the claim data corresponding to the selectedimages. In alternative embodiments, the client may not make anyselection, and the client computing device 104 may generate and displayan average or other statistical analysis of all claim data correspondingto images selected by the comparison image selector 136. In someembodiments, the client may select, via the user interface, various waysto group the images and claim data corresponding to similarity scoresabove the threshold. For instance, the client may choose to group theimages and claim data by the particular make, model, and/or year of thevehicle depicted in the image. In this way, the client may determine anestimated damage level, repair cost, or repair time for the damagedvehicle 102.

FIG. 2 is a data flow diagram depicting inputs, operations, and outputsof an exemplary image search engine 200 for selecting and displayingimages similar to an image of a damaged vehicle. The image search engine200 may be used as the image search engine 118 of FIG. 1 , and includesa similarity score generator 202 and a comparison image selector 204that may correspond to the similarity score generator 134 and comparisonimage selector 136, respectively, of FIG. 1 . The image search engine200 may access a damaged vehicle image 206 and a damaged vehicle digitalimage dataset 208 containing a number of images 210, 212, 214 and anumber of corresponding claim data 216, 218, 220. In order to generatesimilarity scores indicating similarity levels between the damagedvehicle image 206 and each of the number of images 210, 212, 214, thesimilarity score generator 202 may analyze and process the damagedvehicle image 206 with each of the images 210, 212, 214. In variousembodiments, each similarity score may be only a single score, or may bea set of scores reflecting the commonality of various differentpredictive image characteristics.

In the example embodiment shown in FIG. 2 , for instance, similarityscore 222 may correspond to the similarity between damaged vehicle image206 and image 210, similarity score 224 may correspond to the similaritybetween damaged vehicle image 206 and image 212, and similarity score226 may correspond to the similarity between damaged vehicle image 206and image 214. Although three images 210, 212, 214 and threecorresponding similarity scores 222, 224, 226 are depicted in FIG. 2 ,other embodiments may have any number of images with correspondingsimilarity scores.

Each similarity score may be based on predictive characteristics incommon between both images, as discussed above with respect to FIG. 1 .For example, if images of vehicles A and B each depict a shatteredwindshield, while images of vehicle C depict a dented bumper (or awindshield that has only been lightly scratched, etc.), the similarityscore generator 202 may generate a higher score for vehicles A and Bthan for vehicles A and C. That is, although images of all threevehicles depict predictive image characteristics (e.g., imagecharacteristics depicting vehicle damage such as a dented bumper and ashatter windshield), the similarity score generator 202 may generate ahigher similarity score for images with the same predictivecharacteristics than for images with different predictivecharacteristics.

Additionally or alternatively, the similarity score may be based on thenumber of predictive characteristics present in both images. Forexample, a higher similarity score may be generated for two images thatshare four of the same predictive characteristics than for two imagessharing only two of the same predictive characteristics. If images ofvehicles D and E both depict a dented front bumper, a cracked headlight,a shattered windshield, and a detached side mirror, for instance, whileimages of vehicle F only depict a cracked headlight and a shatteredwindshield, the similarity score generator may generate a highersimilarity score for vehicles D and E than for vehicles D and F. As amore specific example, the similarity score may generate an “8” on a10-point scale for vehicles D and E, but only a “5” on a 10-point scalefor vehicles D and F. Moreover, how strongly or weakly predictive eachimage characteristic is may also impact the similarity score for a giventwo images.

A comparison image selector 204 may then process the number ofsimilarity scores 222, 224, 226 and output a set 228 of similarityscores above a similarity score threshold and a set 230 of similarityscores below the similarity score threshold. The threshold may bepreset, or a user may input a threshold. In some embodiments, thecomparison image selector 204 may rank the number of similarity scores222, 224, 226 from highest to lowest in order to select a certain numberof most similar images to be displayed in set 228. For instance, thecomparison image selector 204 may select a fixed number of images (e.g.,20 images), or a number of images selected by a user.

As shown in FIG. 2 , for example, similarity score 222 and similarityscore 226 are both above the similarity score threshold. Theircorresponding images 210 and 214 may be displayed to a user via a userinterface, such as user interface 404 discussed below with respect toFIG. 4 . Similarity score 224, on the other hand, is below thesimilarity score threshold, and therefore its corresponding image 212may not be displayed.

Additionally or alternatively, a set 232 of claim data corresponding tothe images selected by the comparison image selector may be displayed ona user interface, such as user interface 406, discussed below withrespect to FIG. 4 . For instance, claim data 216 corresponding todisplayed image 210, claim data 220 corresponding to displayed image214, and any other claim data corresponding with displayed images withsimilarity scores above the threshold, may be displayed. A statisticalanalysis 234 of the displayed claim data 216, 220, such as an average,median, mean, histogram, density plot, and/or any other type ofnumerical or graphical analysis, may further be displayed on the userinterface. Moreover, the statistical analysis 234 may include ananalysis of damage level, repair time, repair cost, and/or any otherclaim data parameter(s).

FIG. 3 is a flow diagram of an exemplary method 300 for estimating thedamage level, repair time, and/or repair cost for a damaged vehicle. Themethod 300 may be implemented by one or more electronic devices, such asclient computing device 104 and/or server 106 of FIG. 1 , for example.The electronic device(s) may support execution of a dedicatedapplication that may implement the functionalities of the method 300.The method 300 may be executed by a processor or processors of theelectronic device(s), for example.

In the method 300, a historical training dataset populated with digitalimages of damaged vehicles and historical claim data related to thedamaged vehicles may be accessed (block 302). The historical trainingdataset (e.g., historical training dataset 120 of FIG. 1 ) may includeone or more images associated with each vehicle and/or claim data.Additionally, the images associated with a given vehicle may depict thatvehicle from various different perspectives, and/or focus on variousdifferent parts of the vehicle. The historical claim data for a givenvehicle may include data about the damage level of the vehicle, therepair time necessary to repair the damage to the vehicle, the cost ofrepairing the vehicle, the make, model, and/or year of the vehicle, thelocation of the damage on the vehicle, the cause of the damage, and/orany other data related to the damaged vehicle.

A deep learning neural network may be trained (block 304) using thehistorical training dataset. The deep learning neural network mayprocess and analyze the historical images of damaged vehicles andassociated historical claim data from the historical training dataset inorder to learn which characteristics of the images are predictive ofcertain claim data characteristics, such as damage level, repair time,repair cost, etc. Additionally or alternatively, the predictivesimilarity model generator may be trained to learn the degree to whicheach of the damaged vehicle image characteristics is predictive ofdamage level, repair time, and/or repair cost (e.g., strongly or weaklypredictive).

As a result of this training, a predictive similarity model may begenerated (block 306). In particular, the predictive similarity modelmay include data and/or rules indicating which claim datacharacteristics are predictive, and/or the degree to which eachcharacteristic is predictive, as indicated by the deep learning neuralnetwork processing of the historical images and historical claim datafrom the historical training dataset. Additionally or alternatively, thepredictive similarity model may use a mathematical formula or algorithmto model how predictive each of the various image characteristics maybe.

A similarity score for a given two images of damaged vehicles may bedetermined (block 308) using the predictive similarity model. The twoimages of damaged vehicles may include both a digital image of a newlydamaged vehicle and a digital image from a dataset of damaged vehicles(e.g., the damaged vehicle digital image dataset 138). In someembodiments and/or scenarios, the dataset may be include a digital imageof at least one damaged vehicle that is not depicted in the historicaltraining dataset (i.e., the dataset of damaged vehicles may be differentfrom the historical training dataset.) Moreover, the similarity scoremay be generated using a similarity score generator into which thepredictive similarity model is incorporated (e.g., similarity scoregenerator 134), and may be based on a weighting of the damaged vehicleimage characteristics (i.e., the degree to which each characteristic ispredictive of damage level, repair time, and/or repair cost).

In some embodiments and/or scenarios, block 308 includes determining aplurality of similarity scores, each corresponding to a digital image ofa newly damaged vehicle and a respective one of a plurality of digitalimages. For example, each similarity score may correspond to the imageof the newly damaged vehicle and a particular image in a dataset ofother damaged vehicles.

It may be determined that the given two digital images satisfy one ormore similarity criteria (block 310) based on the similarity score. Thedetermination may include determining whether the similarity scoreexceeds a similarity score threshold, which may be pre-set or set by auser. In embodiments where a plurality of similarity scores isdetermined at block 308, the determination at block 310 may includefinding one or more digital images from the dataset for which thecorresponding similarity score is above the similarity score thresholdusing an image search engine (e.g., image search engine 118 of FIG. 1 ).

In response to the determination that the given two digital imagessatisfy the similarity criteria, the two digital images may be displayed(block 312), e.g., on a single page. The single page may be a userinterface display, such as user interface 404 discussed below withrespect to FIG. 4 . In some embodiments, the digital images found usingthe image search engine may all be displayed on the single page. Forexample, if multiple images have similarity scores exceeding thesimilarity score threshold, the multiple images may all be displayed onthe page. In some embodiments, the image search engine may cap thenumber of images displayed to a certain set of images associated withtop similarity scores. For instance, in some embodiments, only imagesassociated with the top ten similarity scores may be displayed.Additionally or alternatively, a client or user may have the option toselect one or more of the most visually similar images from among thedisplayed images via the user interface.

The damage level, repair time, and/or repair cost for one of thevehicles depicted in one of the given two damaged vehicle images may bepredicted (block 314) based on a known damage level, repair time, and/orrepair cost for the other vehicle. In some embodiments, the damagecharacterization or prediction may be based on predictive statistics forthe vehicle(s) depicted in the images found by the image search engine,e.g., for all vehicles with similarity scores exceeding a certainthreshold. The damage characterization, prediction and/or predictivestatistics may be displayed via a user interface such as user interface406, discussed below with respect to FIG. 4 . In particular, thepredictive statistics may appear in graphical form, such as a histogramand/or density plot.

FIG. 4 depicts an exemplary set 400 of user interfaces 402, 404, and 406for uploading an image of a damaged vehicle, comparing the image tosimilar images of other damaged vehicles, and displaying a repairestimate for the damaged vehicle, respectively. An electronic device,such as client computing device 104 of FIG. 1 , may be configured todisplay the interface and/or receive selections and inputs via the userinterfaces, where the electronic device may be associated with a clientand/or user. For example, a dedicated application that is configured tooperate on the electronic device may display the user interfaces. Itshould be appreciated that the user interfaces are merely examples andthat alternative or additional content is possible.

The user interface 402 may enable a user to upload an image of a damagedvehicle, such as image 114 of FIG. 1 (e.g., taken using camera 112). Insome embodiments, a user may upload multiple images of the damagedvehicle. The user interface 402 may be further configured to receiveselections in drop-down menus for the year, make, and model of thevehicle in the image. In some embodiments, the user may select or inputadditional or alternative information, such as the location of damage,the cause of damage, or any other information related to the damageand/or the vehicle.

The user interface 404 may display the uploaded image alongside otherdamaged vehicle images for which repair data is known (e.g., images fromthe damaged vehicle digital image dataset 138 of FIG. 1 , found usingimage search engine 118). In particular, interface 404 may be configuredto receive user selections of one or more of the displayed damagedvehicles that are most similar to the uploaded damaged vehicle imagewith respect to the damage shown (and/or with respect to make, model,etc., if the displayed images are not already limited to such vehicles).

The user interface 406 may display a statistical analysis of the claimdata associated with the selected and/or displayed images, such as ahistogram and/or density plot. Additionally, the interface 406 may beconfigured to receive user selections of whether to view total repairhours and/or total repair cost, as shown in FIG. 4 . In otherembodiments, the user interface 406 may display more, fewer, and/ordifferent types of information, such as a qualitative estimate of damagelevel (e.g., “heavy” or “totaled”), for example.

The following additional considerations apply to the foregoingdiscussion. Throughout this specification, plural instances mayimplement operations or structures described as a single instance.Although individual operations of one or more methods are illustratedand described as separate operations, one or more of the individualoperations may be performed concurrently, and nothing requires that theoperations be performed in the order illustrated. These and othervariations, modifications, additions, and improvements fall within thescope of the subject matter herein.

Unless specifically stated otherwise, discussions herein using wordssuch as “processing,” “computing,” “calculating,” “determining,”“presenting,” “displaying,” or the like may refer to actions orprocesses of a machine (e.g., a computer) that manipulates or transformsdata represented as physical (e.g., electronic, magnetic, or optical)quantities within one or more memories (e.g., volatile memory,non-volatile memory, or a combination thereof), registers, or othermachine components that receive, store, transmit, or displayinformation.

As used herein any reference to “one embodiment” or “an embodiment”means that a particular element, feature, structure, or characteristicdescribed in connection with the embodiment is included in at least oneembodiment. The appearances of the phrase “in one embodiment” in variousplaces in the specification are not necessarily all referring to thesame embodiment.

As used herein, the terms “comprises,” “comprising,” “includes,”“including,” “has,” “having” or any other variation thereof, areintended to cover a non-exclusive inclusion. For example, a process,method, article, or apparatus that comprises a list of elements is notnecessarily limited to only those elements but may include otherelements not expressly listed or inherent to such process, method,article, or apparatus. Further, unless expressly stated to the contrary,“or” refers to an inclusive or and not to an exclusive or. For example,a condition A or B is satisfied by any one of the following: A is true(or present) and B is false (or not present), A is false (or notpresent) and B is true (or present), and both A and B are true (orpresent).

In addition, use of “a” or “an” is employed to describe elements andcomponents of the embodiments herein. This is done merely forconvenience and to give a general sense of the invention. Thisdescription should be read to include one or at least one and thesingular also includes the plural unless it is obvious that it is meantotherwise.

Upon reading this disclosure, those of skill in the art will appreciatestill additional alternative structural and functional designs forautomated correspondence management through the principles disclosedherein. Thus, while particular embodiments and applications have beenillustrated and described, it is to be understood that the disclosedembodiments are not limited to the precise construction and componentsdisclosed herein. Various modifications, changes and variations, whichwill be apparent to those skilled in the art, may be made in thearrangement, operation and details of the method and apparatus disclosedherein without departing from the spirit and scope defined in theappended claims.

The patent claims at the end of this patent application are not intendedto be construed under 35 U.S.C. § 112(f) unless traditionalmeans-plus-function language is expressly recited, such as “means for”or “step for” language being explicitly recited in the claim(s). Thesystems and methods described herein are directed to an improvement tocomputer functionality, and improve the functioning of conventionalcomputers.

What is claimed:
 1. A computer-implemented method, comprising: traininga predictive similarity model, using a historical training datasetpopulated with digital images of damaged vehicles and historical claimdata related to the damaged vehicles, to identify damaged vehicle imagecharacteristics that are predictive of one or more of damage level,repair time, or repair cost; determining, using the predictivesimilarity model, a similarity score for a given two digital images ofdamaged vehicles; and based on the similarity score for the given twodigital images, predicting a damage level, repair time, or repair costfor a first damaged vehicle depicted in a first digital image of thegiven two digital images based on a known damage level, repair time, orrepair cost for a second damaged vehicle depicted in a second digitalimage of the given two digital images.
 2. The computer-implementedmethod of claim 1, wherein determining the similarity score for thegiven two digital images of damaged vehicles further comprises weightingthe damaged vehicle image characteristics based on a degree to whicheach characteristic is predictive of at least one of damage level,repair time, or repair cost.
 3. The computer-implemented method of claim1, wherein determining the similarity score for the given two digitalimages of damaged vehicles includes: determining a plurality ofsimilarity scores each corresponding to a digital image of a newlydamaged vehicle and a respective one of a plurality of digital images ina given dataset of damaged vehicles; and the method further comprisingfinding, using an image search engine, a set of one or more digitalimages of damaged vehicles from the given dataset for which thesimilarity score is above a threshold similarity score.
 4. Thecomputer-implemented method of claim 3, wherein the given dataset ofdamaged vehicles is populated with digital images of at least onedifferent damaged vehicle than the historical training dataset.
 5. Thecomputer-implemented method of claim 3, further comprising displayingthe set of digital images found using the image search engine.
 6. Thecomputer-implemented method of claim 5, further comprising displayingone or more of a damage level, a repair time, or a repair costassociated with the one or more digital images of damaged vehicles foundusing the search engine.
 7. The computer-implemented method of claim 3,further comprising: generating predictive statistics for a damage level,repair time, or repair cost for the newly damaged vehicle based on adamage level, repair time, or repair cost for damaged vehicles depictedin the set of one or more digital images of damaged vehicles found usingthe image search engine; and displaying the predictive statistics.
 8. Atangible, non-transitory computer-readable medium storing instructionsthat, when executed by one or more processors, cause the one or moreprocessors to: train a predictive similarity model, using a historicaltraining dataset populated with digital images of damaged vehicles andhistorical claim data related to the damaged vehicles, to identifydamaged vehicle image characteristics that are predictive of one or moreof damage level, repair time, or repair cost; determine, using thepredictive similarity model, a similarity score for a given two digitalimages of damaged vehicles; and based on the similarity score for thegiven two digital images, predict a damage level, repair time, or repaircost for a first damaged vehicle depicted in a first digital image ofthe given two digital images based on a known damage level, repair time,or repair cost for a second damaged vehicle depicted in a second digitalimage of the given two digital images.
 9. The tangible, non-transitorycomputer-readable medium of claim 8, wherein the instructions todetermine the similarity score for the given two digital images ofdamaged vehicles further include instructions to weight the damagedvehicle image characteristics based on a degree to which eachcharacteristic is predictive of at least one of damage level, repairtime, or repair cost.
 10. The tangible, non-transitory computer-readablemedium of claim 8, wherein: the instructions to determine the similarityscore for the given two digital images of damaged vehicles furtherinclude instructions to determine a plurality of similarity scores eachcorresponding to a digital image of a newly damaged vehicle and arespective one of a plurality of digital images in a given dataset ofdamaged vehicles; and wherein the instructions further cause the one ormore processors to find, using an image search engine, a set of one ormore digital images of damaged vehicles from the given dataset for whichthe similarity score is above a threshold similarity score.
 11. Thetangible, non-transitory computer-readable medium of claim 10, whereinthe given dataset of damaged vehicles is populated with digital imagesof at least one different damaged vehicle than the historical trainingdataset.
 12. The tangible, non-transitory computer-readable medium ofclaim 10, wherein the instructions further include instructions todisplay the set of digital images found using the image search engine.13. The tangible, non-transitory computer-readable medium of claim 12,further including instructions to display one or more of a damage level,a repair time, or a repair cost associated with the one or more digitalimages of damaged vehicles found using the search engine.
 14. Thetangible, non-transitory computer-readable medium of claim 10, furtherincluding instructions to: generate predictive statistics for a damagelevel, repair time, or repair cost for the newly damaged vehicle basedon a damage level, repair time, or repair cost for damaged vehiclesdepicted in the set of one or more digital images of damaged vehiclesfound using the image search engine; and display the predictivestatistics.
 15. A computer system comprising: one or more processors;and a memory storing instructions that, when executed by the one or moreprocessors, cause the one or more processors to: train a predictivesimilarity model, using a historical training dataset populated withdigital images of damaged vehicles and historical claim data related tothe damaged vehicles to train a neural network, to identify damagedvehicle image characteristics that are predictive of one or more ofdamage level, repair time, or repair cost; determine, using thepredictive similarity model, a similarity score for a given two digitalimages of damaged vehicles; and based on the similarity score for thegiven two digital images, predict a damage level, repair time, or repaircost for a first damaged vehicle depicted in a first digital image ofthe given two digital images based on a known damage level, repair time,or repair cost for a second damaged vehicle depicted in a second digitalimage of the given two digital images.
 16. The computer system of claim15, wherein the instructions to determine the similarity score for thegiven two digital images of damaged vehicles further includeinstructions to weight the damaged vehicle image characteristics basedon a degree to which each characteristic is predictive of at least oneof damage level, repair time, or repair cost.
 17. The computer system ofclaim 15, wherein: the instructions to determine the similarity scorefor the given two digital images of damaged vehicles further includeinstructions to determine a plurality of similarity scores eachcorresponding to a digital image of a newly damaged vehicle and arespective one of a plurality of digital images in a given dataset ofdamaged vehicles; and wherein the instructions further cause the one ormore processors to find, using an image search engine, a set of one ormore digital images of damaged vehicles from the given dataset for whichthe similarity score is above a threshold similarity score.
 18. Thecomputer system of claim 17, wherein the given dataset of damagedvehicles is populated with digital images of at least one differentdamaged vehicle than the historical training dataset.
 19. The computersystem of claim 17, wherein the instructions further includeinstructions to display the set of digital images found using the imagesearch engine.
 20. The computer system of claim 19, further includinginstructions to display one or more of a damage level, a repair time, ora repair cost associated with the one or more digital images of damagedvehicles found using the search engine.